Several applications for image editing allow the user to associate an enhancement or an image filter, such as a color change, with a color range. These techniques can be found in such programs as Nik Software's Color Efex Pro′ and Nikon's Capture NX2™. For instance, these products contain tools allowing the user to darken all blue pixels in the image. Example tools are Channel Mixer, Select Color Range, or Hue/Saturation in Adobe® Photoshop®, Contrast Color Range in Nik Color Efex Pro™, or LCH Editor in Nikon Capture NX2™.
A common feature of these tools is that the strength of an effect is determined by the color of a pixel. The disadvantage of this approach is that small local variations among pixel values, resulting from noise and compression artifacts, can lead to larger variations in effect strength, because the small local variations are typically amplified during image editing routines.
An example filter expression is g(x)=T[ƒ(x)], where ƒ(x) is a function of x, g(x) is the filtered function, and T is the filter function. The function ƒ(x) is considered a one-dimensional image because it depends on one variable. More commonly, image-editing functions filter two-dimensional images. An example two-dimensional image filter expression isg(x,y)=T[ƒ(x,y)]where ƒ(x, y) is a two-dimensional original image to be filtered, g(x, y) is the filtered image, and T is the filter function. Here, filter inputs and outputs, such as ƒ or g, typically represent an image consisting of a plurality of pixels. Any specific individual pixel may be referred to by specifying the underlying coordinates, as in ƒ(x0, y0) is the pixel at coordinates (x0, y0) in the image ƒ.
An example filter g=T[ƒ] may perform a darkening operation. There are many ways to express a brightness change filter based on a color in a simple form, but here, a general formulation convenient for explaining the invention is used. The filtering operation may darken colors in a color range near a user-selected color, c. As a result of filtering, pixels with colors similar to the color c receive a strong darkening effect, pixels with colors dissimilar to c do not receive a darkening effect, and a smooth transition between darkened areas and areas with no darkening is desired.
A “color distance” is an inverse measure of color similarity between two pixels, with more similarly colored pixels having a smaller color distance. More precisely, a color distance, Dc, is a function that assigns a non-negative value to the difference in the color of two pixels. Many such distance, or difference, functions are known in the art, and have the characteristics that for any three pixels p, q, and r:Dc(p,q)≧0 with Dc(p,q)=0 if and only if pixels p and q are the same color,Dc(p,q)=Dc(q,p), andDc(p,q)+Dc(p,r)≧Dc(p,r).
One suitable function measuring color distance between the color of a pixel ƒ(x0,y0), denoted as “color[ƒ(x0, y0)]”, and the color c is:Dc=∥color[ƒ(x0,y0)]−c∥2,  (1)where color[ƒ(x0,y0)] and c are color values in an appropriate color reference system and ∥r∥2 denotes the l2 norm of r, as explained further below. Appropriate color reference systems include grey-scale and color reference systems such as RGB, YCrCb, or Lab color systems, where a color c is expressed as a three-coordinate vector of values. In an RGB system, for example, c is expressed as a three-coordinate vector, c=(c[r], c[g], c[b]), where c[r] is the red coordinate, c[g] is the green coordinate, and c[b] is the blue coordinate. If ƒ(x0, y0)=(ƒ[r], ƒ[g], ƒ[b]), for example, then a suitable norm isDc[ƒ(x0,y0),c]={(c[r]−ƒ[r])2+(c[g]−ƒ[g])2+(c[b]−ƒ[b])2}0.5.The norm above is known in the art as an l2 norm. Alternatively, color distance may be expressed as an l1 norm, given byDc[ƒ(x0,y0),c]=∥color[ƒ(x0,y0)]−c∥1=|c[r]−ƒ[r]|+|c[g]−ƒ[g]|+|c[b]−ƒ[b]|.
Let the function s[ƒ(x0, y0), c] denote a measure of similarity of color of the pixel ƒ(x0, y0) and the color c, such ass[ƒ(x0,y0),c]=1/(1+Dc[ƒ(x0,y0),c]).  (2)In Equation 2, when s equals 1 there is a maximum similarity of color, and s equals zero represents a lower limit for color similarity.
A filtering effect based on similarity of color at any pixel (x, y) can be written using a filtering functiong(x,y)=ƒ(x,y)+Δh s[ƒ(x,y),c],  (3)with Δh representing a maximum change in pixel color value. The value Δh can be a scalar value if monochromatic brightness changes are desired, or it can be a color triple if a color modifying filter is desired. Note that, in typical reference color systems for digital images, darker colors have lower pixel color values, and, because the example filter produces a darkening effect, a scalar valued Δh is less than zero for darkening.
When Δh=(Δh[r], Δh[g], Δh[b]) is an RGB color, the example darkening filter becomes an example color modifying filter, modifying colors in the image. Even colorizing operations can be performed in the image, such as when Δh[r]>Δh[g]>Δh[b], for example, a filter warming up the area will be performed. Other usages of Δh are discussed below. For purposes of the explanation that follows, the terms “warming filter,” “darkening filter,” and “brightness changing filter,” may be used as species of “color changing filters.”
Also note that, in a typical digital image, pixel color values are scaled between a value of zero and a maximum value, MAX, dependent on the number of bits representing a pixel color. Typically, when colors are represented by n-bit non-negative numbers, the maximum color value, MAX, is given by MAX=2n−1. Note that a value lower than zero or higher than MAX can occur in Equation 3 and needs to be compensated for in actual code, but can be trivially corrected by saturating the formula at a lower value of zero and an upper value of MAX, as known to those skilled in the art.
An alternative change that a user may wish to perform at any pixel (x, y) is to increase or decrease the contrast in a certain area, usingg(x,y)=ƒ(x,y)+Δk s[ƒ(x,y),c][ƒ(x,y)−ƒ(x,y)]  (4)where Δk denotes the desired contrast change, and ƒ(x, y) denotes a lowpass filtered version of the image ƒ(x, y). In Equation 4, a value of Δk equal to zero causes no contrast change, and a positive (negative) value of Δk causes an increase (decrease) in contrast.
The filter used to produce ƒ(x, y) here is not limited to a Gaussian blur. The filter may be any filter that passes low-frequency components in the image while suppressing high-frequency components, a filter known as a lowpass filter. The values from the term [ƒ−ƒ] on the right of Equation 4 then represents a high-pass filtering of the structure information in the image, i.e., the image with low frequency components suppressed, and therefore Equation 4 can be interpreted as a color-weighted increase of high-frequency structure information.
The color-changing and contrast-changing effects of Equation 3 and Equation 4 may be combined using the formulag(x,y)=ƒ(x,y)+{Δh+Δk[ƒ(x,y)−ƒ(x,y)]}s[ƒ(x,y),c]  (5)The examples above relate to a filter that changes regions in an image that are represented by a single color. A simple version of a polarization filter, darkening and adding contrast to sky-blue areas, or a typical skin-brightening filter may be implemented using Equation 5.
However, there is an inherent disadvantage in the formulations of Equations 3 through 5, as this method is not robust in the presence of noise. Consider, for instance, an image with a large blue-sky area whose mean color value is the color c. If the image is shot with a high ISO number and with strong JPG compression, not all pixels in the sky will be equally blue. In this situation, the similarity expression s[ƒ(x, y), c] reflects the noise present in a blue-sky pixel to a certain extent. In the worst case, where variations in the color are due only to noise but the underlying colors are very similar, the function s modifies the noise amplitude to the extent ofsimilarity_noise_amplitude=original_noise_amplitude*max|s|. In example Equations 3-5, the change is added to the original image. The resulting filtered image g has amplified noise, with {1+max|s|} times the amplitude of what it was in the original image. The noise increase is also based upon the values of Δh and Δk, which is characteristic to most image editing filters. The increase in noise level will frequently be visible to the observer of the image, and visibly degrades the image.
Conventional approaches to the noise enhancement problem are (1) to first reduce the noise in the original image and then perform the filtering function, or (2) to first perform the filtering function and then reduce the noise in the resulting image. These prior art approaches have the disadvantage that actual desired features in the image are being reduced.
An alternative prior art approach is to apply noise reduction selectively to the affected areas, but this also produces unwanted side effects, such as loss of detail, unwanted artifacts, or an uneven distribution of image structure.